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STORAGE MEDIUM FOR RECORDING SUBTITLE INFORMATION BASED ON TEXT 
CORRESPONDING TO AV DATA HAVING MULTIPLE PLAYBACK ROUTES, REPRODUCING 

APPARATUS AND METHOD THEREFOR 

CROSS-REFERENCE TO RELATED APPLICATIONS 

[0001] This application claims the benefit of PCT International Patent Application 
No. PCT/KR2004/002481, filed September 24, 2004, Korean Patent Application No. 2004-4442, 
filed January 20, 2004, in the Korean Intellectual Property Office, and Korean Patent Application 
No. 2003-67919, filed September 30, 2003, in the Korean Intellectual Property Office, the 
disclosures of which are incorporated herein by reference. 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

[0002] Aspects of the present invention relate to reproduction of a storage medium recording 
moving picture data, and more particularly, to a storage medium recording subtitle information 
compatible with a subpicture method of a digital versatile disc (DVD) and a presentation method 
of a bluray disc, and based on text corresponding to moving picture data having a plurality of 
playback routes. Other aspects of the invention relate to an apparatus for and a method of 
reproducing a moving picture from the moving picture data on the storage medium. 

2. Description of the Related Art 

[0003] Among conventional text-based subtitle technologies are Synchronized Accessible 
Media Interchange (SAMI) technology of Microsoft Corporation and Real-text technology of 
RealNetwork, Inc.. These conventional text-based subtitle technologies are based on a video 
stream data file recorded on a storage medium or a video stream data file provided on a 
network, and have a structure in which a subtitle is output based on synchronization time 
information with video data. 

[0004] FIG.1 is a reference diagram explaining the conventional text-based subtitle 
technology. Referring to FIG.1, a text-based subtitle file, corresponding to video stream data, 
and based on synchronization time information is included in a subtitle file. For example, at 
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<sync time 00:05>, a subtitle corresponding to video stream data is output. That is, the 
conventional text-based subtitle information provides a subtitle file structure constructed 
assuming that continuous video data is reproduced. 

[0005] FIG. 2 is a reference diagram explaining problems of the conventional text-based 
subtitle technology. Referring to FIG. 2, with the conventional text-based subtitle file structure, 
when a multiple story video having a plurality of playback routes is reproduced according to a 
user input, processing subtitles according to alternate routes of the multiple story video is 
impossible using the synchronization time information as shown in FIG. 2. That is, only story 
route A can be reproduced, and story route B cannot be reproduced. 

[0006] In addition, the conventional text-based subtitle data is not compatible with subtitle 
data based on a bitmap image such as a subpicture method of a DVD or a presentation method 
of a bluray disc. 

SUMMARY OF THE INVENTION 

[0007] To solve the above and/or other problems, aspects of the present invention provide a 
storage medium recording subtitle information compatible with a subpicture method of a DVD 
and a presentation method of a bluray disc, based on text corresponding to moving picture data 
having a plurality of playback routes. Additional aspects of the invention provide an apparatus 
for reproducing and a method of reproducing compatible with the subpicture method of the DVD 
and the presentation method of the bluray disc. 

[0008] According to an aspect of the present invention a storage medium includes: moving 
picture data having a plurality of playback routes; a plurality of subtitle data items corresponding 
to the playback routes and supporting random search for a subtitle; and mapping information 
linking the moving picture data and the subtitle data. 

[0009] The subtitle data may include reference offset information indicating reference 
information to randomly search for a subtitle of a desired time at a high speed and reproduce 
the subtitle, text data containing subtitle contents that are converted into pixel data and output, 
style information specifying an output style of the pixel data, and control information to control 
the output of the converted pixel data. 
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[0010] By using information on a time elapsed from the reference offset information, the 
subtitle of a desired time may be randomly searched for at a high speed among the subtitle 
data. The reference offset information may include a cell identifier (VOBJD and CELLJD) of a 
video object that is a recording unit of the storage medium, or a start position of a clip that is a 
recording unit of the storage medium. 

[0011] The subtitle data may be described in a form of a markup language or a form of a 
binary table. In the subtitle data described in the form of the binary table, the style information, 
the control information, and the text information may have respective identifiers for 
distinguishing each other, and the control information may include indication information 
indicating the style information and the text information corresponding to the control information, 
and in order to reduce a search time by integer calculation when random search is performed, 
the size of each of the style information and the control information may be predetermined and 
sequentially recorded in a predetermined area. 

[0012] The style information may include at least one among information items on the width 
and height of the pixel data area, a background color, a time when the pixel data is to be stored 
and deleted in a buffer memory, a starting point from which subtitle text is rendered, line 
spacing, output direction, bold type and Italic type of subtitle text, line break, color of subtitle 
text, and information on character code encoding. As used herein, an expression of the form at 
least one among A, B, and C includes A or B or C, A and B, A and C, B and C, and A and B 
and C. 

[0013] The control information may include at least one among information items on an area 
on which the pixel data is to be output on the entire screen, a start point of subtitle text in the 
pixel data output area, and a synchronization time indicating when the pixel data is to appear 
and disappear in synchronization with the moving picture data. 

[0014] The synchronization time information may be expressed as a lapse time from a 
reference cell (CELL) of a video object (VOBU) that is reference offset information of the moving 
picture data, or as a lapse time from a start position of a clip that is reference offset information 
of the moving picture data. 
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[0015] The synchronization time information may be expressed by using a present time 
stamp (PTS) time based on a reference time for reproducing moving pictures. 

[0016] The subtitle data or mapping information may further include at least one among font 
information describing the font of subtitle data to be displayed on the screen, information on a 
producer making the subtitle, packet identifier (PID) information of the subtitle data to distinguish 
from the moving picture data, and subtitle indication information by language of the subtitle data. 

[0017] According to another aspect of the present invention, a reproducing apparatus for 
reproducing a storage medium on which moving picture data is recorded includes: a decoder 
decoding moving picture data having a plurality of playback routes; and a subtitle processor 
converting subtitle data, corresponding to a predetermined route and selected by using a 
plurality of subtitle data items corresponding to the playback routes and supporting random 
search for a subtitle and mapping information linking the moving picture data and the subtitle 
data, into pixel data, synchronizing the converted pixel data with the moving picture data, and 
outputting the pixel data. 

[0018] The subtitle processor may include: a text subtitle decoder identifying subtitle data 
corresponding to the moving picture data of a route to be reproduced by parsing the mapping 
information, and converting the identified subtitle data into pixel data by parsing the subtitle 
data; and a graphic controller controlling the pixel data by using the parsed mapping information 
and subtitle data such that the pixel data is synchronized with the moving picture data and 
output. 

[0019] In the subtitle data, by decoding the subtitle data, the text subtitle decoder may parse 
reference offset information indicating reference information to randomly search for a subtitle of 
a desired time at a high speed and reproduce the subtitle, text data containing subtitle contents 
that are converted into pixel data and output, style information specifying an output style of the 
pixel data, and control information to control the output of the converted pixel data, and based 
on the style information, convert the text data into pixel data, and by using the parsed control 
information, the graphic controller may control such that the converted pixel data is 
synchronized with the moving picture data and output. 
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[0020] The text subtitle decoder may randomly search the subtitle data for the subtitle of a 
desired time at a high speed by using information on a time elapsed from the reference offset 
information. 

[0021] By using synchronization time information expressed as a lapse time from a reference 
cell (CELL) of a video object (VOBU) that is reference offset information of the moving picture 
data, or as a lapse time from a start position of a clip that is reference offset information of the 
moving picture data, the graphic controller may control the converted pixel data to be 
synchronized with the moving picture data and output. 

[0022] According to an aspect of the present invention, another method of reproducing data 
on a storage medium storing moving picture data having a plurality of playback routes, a 
plurality of subtitle data items corresponding to the playback routes and supporting random 
search for a subtitle, and mapping information linking the moving picture data and the subtitle 
data, includes: reading the subtitle data corresponding to moving picture data of a route to be 
reproduced by parsing the mapping information; identifying subtitle data of a position to be 
reproduced according to continuous reproduction or reproduction by random search, by parsing 
the subtitle data, and converting the subtitle data into pixel data; and synchronizing the 
converted pixel data with the moving picture data and outputting the pixel data. 

[0023] By using the mapping information and the subtitle data file structure according to the 
aspects of the present invention, subtitle information compatible with a bitmap image method 
such as the subpicture method of a DVD and the presentation method of a bluray disc, and 
based on text corresponding to moving picture data having a plurality of playback routes, can be 
provided. 

[0024] Further, by including reference offset information in a subtitle data file, when random 
search or reproduction is performed, the time for searching for a subtitle can be reduced such 
that search efficiency is enhanced. 

[0025] Additional aspects and/or advantages of the invention will be set forth in part in the 
description which follows and, in part, will be obvious from the description, or may be learned by 
practice of the invention. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

[0026] These and/or other aspects and advantages of the invention will become apparent 
and more readily appreciated from the following description of the embodiments, taken in 
conjunction with the accompanying drawings of which: 

FIG. 1 is a reference diagram explaining a conventional text-based subtitle technology; 

FIG. 2 is a reference diagram explaining problems of the conventional text-based subtitle 
technology illustrated in FIG. 1; 

FIG 3 is a reference diagram explaining text-based subtitle information corresponding to 
moving picture data having a plurality of playback routes according to an embodiment of the 
present invention; 

FIG. 4 is a diagram explaining mapping information of moving picture data and subtitle 
data having the plurality of playback routes shown in FIG. 3; 

FIG. 5 is a diagram explaining subtitle indication information by language corresponding 
to moving picture data having the plurality of playback routes; 

FIG 6 is a diagram explaining a structure of subtitle data; 

FIG. 7 is a diagram explaining a usage example of each field of subtitle data having the 
data structure shown in FIG 6; 

FIG. 8 is a diagram explaining a structure of a reproducing apparatus according to an 
embodiment of the present invention; 

FIG. 9 is a diagram explaining a structure of the subtitle processor shown in FIG. 8; 

FIG. 10 is a flowchart of a method of reproducing subtitle information based on text 
corresponding to moving picture data having a plurality of playback routes according to an 
embodiment of the present invention; 

FIG 11 is a diagram explaining the data structure of mapping information shown in FIG. 
4 implemented on a DVD disc; 

FIG. 12 is a diagram showing an example of the mapping information shown in FIG. 11 
implemented by using a markup language on a DVD disc; 

FIG. 13 is a diagram showing an example of the mapping information shown in FIG. 11 
described in the form of a table implemented on a DVD disc; 

FIG. 14 is a diagram explaining the structure of subtitle data shown in FIG. 6 
implemented on a DVD disc; 
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FIGS. 15A and 15B are diagrams showing an example of the subtitle data shown in FIG. 
6 implemented on a DVD disc displayed on a screen; 

FIG. 16 is a diagram showing an example of the subtitle data shown in FIG. 6 
implemented by a markup language on a DVD disc; 

FIG 17 is a diagram showing an example of the subtitle data shown in FIG. 6 
implemented in a binary table on a DVD disc; 

FIG. 18 is a diagram explaining the data structure of the mapping information shown in 
FIG. 4 implemented on a bluray disc; 

FIG. 19 is a diagram showing an example of the mapping information shown in FIG. 18 
implemented by using a markup language on a bluray disc; 

FIG. 20 is a diagram showing an example of the mapping information shown in FIG 18 
implemented in a table on a bluray disc; 

FIG. 21 is a diagram explaining the structure of the subtitle data shown in FIG 6 
implemented on a bluray disc; 

FIGS. 22A and 22B are diagrams showing an example of the subtitle data shown in FIG. 
6 implemented on a bluray disc displayed on a screen; 

FIG. 23 is a diagram showing an example of the subtitle data shown in FIG. 6 
implemented by a markup language on a bluray disc; and 

FIG 24 is a diagram showing an example of the subtitle data shown in FIG. 6 
implemented in a binary table on a bluray disc. 

DETAILED DESCRIPTION OF THE EMBODIMENTS 

[0027] Reference will now be made in detail to the present embodiments of the present 
invention, examples of which are illustrated in the accompanying drawings, wherein like 
reference numerals refer to the like elements throughout. The embodiments are described 
below in order to explain the present invention by referring to the figures. 

[0028] FIGS. 3 through 7 are diagrams explaining mapping information of moving picture 
data and subtitle data having a plurality of playback routes and the structure of subtitle data 
according to an embodiment of the present invention. 
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[0029] FIGS. 8 and 9 are diagrams explaining a structure of a reproducing apparatus 
according to an embodiment of the present invention and FIG. 10 is a flowchart of a method of 
reproducing subtitle data according to an embodiment of the present invention 

[0030] FIGS. 11 through 17 are diagrams explaining an example of the mapping information 
and subtitle data implemented on a DVD disc. 

[0031] FIGS. 18 through 24 are diagrams explaining an i example of mapping information 
and subtitle data implemented on a bluray disc. 

[0032] A storage medium recording text-based subtitle information corresponding to moving 
picture data having a plurality of playback routes, and a reproducing apparatus and a 
reproducing method therefor will now be explained. The storage medium comprises a structure 
including moving picture data having a plurality of playback routes for multiple story playback, a 
plurality of subtitle data items corresponding to the plurality of playback routes and supporting 
random search for a subtitle, and mapping information linking the moving picture data and the 
subtitle data. 

[0033] Referring to FIG. 3, in order to select and reproduce story A route or story B route for 
multiple story playback, video data A and video data B are linked to video data C. 

[0034] FIG. 4 shows mapping information for linking moving picture data having a plurality of 
playback routes and a plurality of subtitle data corresponding to the moving picture data in order 
to support multiple story playback. The mapping information includes Subtitle Indication 
Information for subtitle data items A, B and C, corresponding to video data items, A, B and C, 
respectively,. Accordingly, where a user selects between playback of the story A route and 
playback of the story B route, the video data and the subtitle data corresponding to the selection 
may be output continuously. 

[0035] FIG. 5 is a diagram explaining Subtitle Indication Information by language 
corresponding to moving picture data having a plurality of playback routes. Referring to FIG. 5, 
subtitle indication information by language indicates related mapping information for individual 
subtitle languages supported. Further, the mapping information indicates a plurality of subtitle 
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data items corresponding to a plurality of playback routes, respectively, as described above with 
reference to FIG. 4. 

[0036] For example, as shown in FIG. 5, if in a storage medium providing English, Korean, 
and Japanese subtitles, the Korean subtitle is selected by a user selection, or is set as a default, 
subtitle/video mapping information Y supporting the Korean subtitle is linked by a pointer of 
related mapping indication information. Also, a plurality of the subtitle data A, B and C written in 
the Korean language, are linked to A, B and C video data items having a plurality of playback 
routes included in subtitle/video mapping information Y, respectively. Accordingly, multiple 
language subtitles are supported and at the same time, multiple story playback of moving 
picture data having a plurality of playback routes is enabled. 

[0037] FIG. 6 is a diagram explaining a structure of subtitle data according to an embodiment 
of the present invention. FIG. 6 shows the structure of a plurality of subtitle data linked by 
related subtitle indication information included in the subtitle/video mapping information 
described above with reference to FIG. 4. The subtitle data includes header information, 
reference offset information, time information and/or text information. The header information is 
information on the entire subtitle data and may include position information of a subtitle to be 
displayed on a screen, and international standard record code (ISRC) as information on a 
producer or a support language code. 

[0038] The reference offset information is reference information for randomly searching for a 
subtitle of a desired time at a high speed and reproducing the subtitle. A cell identifier (VOB-ID 
and CELL J D) of a video object (VOBU) used as a recording unit of a DVD storage medium, or 
the starting point of a clip used as a recording unit of a bluray disc storage medium may be used 
as the reference offset information. 

[0039] Synchronization time information synchronizing a subtitle with video data may be 
expressed as a lapse time from a starting point of a reference cell of a VOBU or a clip that is 
reference offset information of the moving picture data. In particular, synchronization time 
information may be expressed by using a presentation time stamp (PTS) time based on a 
reference time for reproducing moving pictures. 
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[0040] Text information indicates text data containing subtitle contents to be converted into 
pixel data to present a bitmap image of the text subtitle. The text information may include style 
information specifying an output style of pixel data and control information to control the output 
of the converted pixel data. The style information and control information will be explained later. 

[0041] FIG. 7 is a diagram explaining a usage example of each field of subtitle data having 
the data structure of FIG. 6. The field subtitle data shown in FIG 7 includes position information 
of a subtitle to be displayed on a screen, a language code and/or ISRC information as header 
information. The subtitle data includes a plurality of synchronization time information items 
synchronizing the subtitle with video data, and a plurality of text information items of a subtitle to 
be used as subtitle contents. Further, the subtitle data includes reference offset information for 
random search for a subtitle, and each reference offset information item indicates an immediate 
next reference offset information item. Accordingly, by using reference offset information, a 
subtitle may be searched for at a high speed and synchronization time information for 
synchronizing a subtitle with video data can also be expressed by a lapse time from reference 
offset information. 

[0042] The storage medium described above with respect to FIGS. 3-7 includes moving 
picture data having a plurality of playback routes, a plurality of subtitle data corresponding to the 
plurality of playback routes, and subtitle/video mapping information for multiple story playback. 
The subtitle data includes reference offset information for high speed random search. In order 
to support multiple language subtitles, subtitle indication information by language may also be 
included. The subtitle/video mapping information, subtitle data, and subtitle indication 
information by language above described may be recorded as files separate from the moving 
picture data, and may be recorded on a same storage medium on which moving picture data is 
recorded or may be recorded separately in a server or the like on the Internet. 

[0043] Based on the data structure of the storage medium described above, an apparatus for 
reproducing the storage medium will now be described. FIG. 8 is a diagram explaining a 
structure of a reproducing apparatus according to an embodiment of the present invention. 

[0044] The reproducing apparatus shown in FIG. 8 includes a reading unit 810, a decoder 
830, a subtitle processor 840, and a blender 850. Also, the reproducing apparatus may further 
include a buffer unit 820 temporarily storing data to buffer the data. 
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[0045] The reading unit 810 reads data items having the data structure described above from 
a storage medium or from the Internet 800, and depending on a type of each data item, stores 
the data in a corresponding buffer unit 820. That is, AV stream data is stored in a moving 
picture data buffer 822, a plurality of subtitle data items corresponding to a plurality of playback 
routes (Refer to FIG. 6) are stored in a subtitle data buffer 824, and subtitle/video mapping 
information (Refer to FIG. 4) and subtitle indication information by language (Refer to FIG. 5) are 
stored in a corresponding buffer 826. Further, font information of a subtitle is stored in a default 
font buffer 828. A resident font buffer 842 storing a resident font read from the storage medium 
or from the Internet and residing on a memory may also be included. 

[0046] The decoder 830 decodes video stream data stored in the moving picture data buffer 
822 and transmits moving pictures to be output to the blender 850. The subtitle processor 840 
generates a graphic image of a subtitle to be overlaid on moving pictures to be output, and 
controls the output of a subtitle to be synchronized with video data. That is, the subtitle 
processor 840 receives subtitle data from the subtitle data buffer 824, receives mapping 
information from the subtitle/video mapping information buffer 826, and identifies subtitle data 
corresponding to the moving picture data corresponding to a selected language and a selected 
story route. By decoding the subtitle data, the subtitle processor 840 converts (or renders) the 
subtitle data into pixel data for a bitmap image and transmits the pixel data to the blender 850. 
By using style information included in the subtitle data, the output style of the subtitle may be 
controlled and by using control information included in the subtitle data, an output time of the 
subtitle data may be controlled to be synchronized with the video data. 

[0047] Font information stored in the default font buffer 828 and/or the resident font buffer 
842 may be used. A detailed structure of the subtitle processor 840 is explained below. 

[0048] Rendering in the present invention indicates all required activities related to converting 
subtitle text data into graphic data to express the text data on a display apparatus. That is, the 
rendering indicates all processes for finding a font matching a character code of each character 
in text data, from download font data read from an information storage medium, or from the 
resident font data, converting into a graphic, and outputting on a screen. 
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[0049] The blender 850 receives decoded video data from the decoder 830, receives 
converted pixel data from the subtitle processor 840, overlays the pixel data of a subtitle on an 
image based on the video data, and outputs to the display apparatus 860. 

[0050] FIG. 9 is a diagram explaining the structure of the subtitle processor 840 of FIG. 8 in 
detail. Referring to FIG 9, the subtitle processor 840 includes a text subtitle decoder 902 and a 
graphic controller 904. 

[0051] The text subtitle decoder 902 receives subtitle data, subtitle/video mapping 
information, other language information, and font information from the buffers 824 and 826, 
parses the subtitle/video mapping information, identifies subtitle data corresponding to the 
moving picture data of a route to be reproduced, parses the identified subtitle data, and converts 
the identified subtitle data into pixel data for a bitmap image graphic. 

[0052] By using the subtitle/video mapping information and subtitle data parsed by the text 
subtitle decoder 902, the graphic controller 904 controls an output time and position of the 
converted pixel data such that the converted pixel data is output synchronized with the moving 
picture data. 

[0053] More specifically, the text subtitle decoder 902 decodes subtitle data, and parses 
reference offset information indicating reference information for randomly searching at a high 
speed for a subtitle of a desired time among subtitle data and reproducing the subtitle; text data 
including subtitle contents to be converted into pixel data and then output; style information 
specifying an output style of pixel data; and control information for controlling the output of the 
converted pixel data. Based on the parsed style information, text data is converted into pixel 
data. 

[0054] The graphic controller 904 controls based on the parsed control information to output 
the converted pixel data synchronized with moving picture data. An output time point or an 
output position may be included in the control information. 

[0055] A method of reproducing subtitle data based on the data structure recorded on a 
storage medium and the structure of the reproducing apparatus described above will now be 
explained. 
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[0056] FIG. 10 is a flowchart of a method of reproducing text-based subtitle information 
corresponding to moving picture data having a plurality of playback routes according to an 
embodiment of the present invention. Referring to FIG. 10, in order to support multiple language 
subtitles and multiple story playback, a subtitle indication information item corresponding to a 
language selected by a user or set as a default in a reproducing apparatus is read among 
subtitle indication information by language (Refer to FIG. 5) in operation 1002. By parsing 
subtitle indication information corresponding to the selected language, linked subtitle/video 
mapping information is read in operation 1004. By parsing the read subtitle/video mapping 
information, subtitle data linked to video data of a route to be reproduced is read in operation 
1006. Based on style information and text information included in the read subtitle data, pixel 
data for a bitmap image graphic is generated in operation 1008. By controlling a time and 
position for outputting a subtitle based on control information included in the subtitle data, the 
generated pixel data is output in operation 1010. 

[0057] Accordingly, the subtitle of the language selected by the user or as a default setting 
may be reproduced along the story playback route. Subtitle indication information by language, 
subtitle/video mapping information, or subtitle data may be recorded on a same storage medium 
on which moving picture data is recorded or may be disposed separately, such as for example, 
on the Internet at an Internet address stored on the storage medium. 

[0058] Referring to FIGS. 11 through 17, an example of implementing the mapping 
information and subtitle data on a DVD disc will now be explained. 

[0059] On a DVD disc, a video manager (VMG) in which header information on the entire 
video title is recorded and a plurality of video title sets (VTS) are stored. A video object set 
(VOBS) is linked to the VMG. A VOBS for displaying a menu screen and a plurality of video 
objects (VOBs) are linked to the VTS. A VOB is formed with a plurality of Cells, and each cell is 
formed with a plurality of video object units (VOBUs). 

[0060] In information for navigation of video data, VOBU_VOB_IDN information and 
VOBU_C_IDN information are recorded such that a video object to which a current VOBU unit 
belongs and identifier of a Cell (VOBJD and CELLJD) may be known. Also, cell elapsed time 
(C_ELTM) information is recorded such that a relative reproduction time from the first video 
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frame of a Cell to which a current VOBU belongs, to the first video frame of the current VOBU 
may be known. 

[0061] Mapping information and the structure of subtitle data based on the DVD disc 
structure described above will now be explained. FIG. 11 is a diagram explaining the data 
structure of mapping information shown in FIG. 4 implemented on a DVD disc. Referring to FIG. 
11 , the DVD disc includes a video object VOB that is a recording unit of moving picture data of a 
DVD and indication information on subtitle data linked to the VOB. 

[0062] Subtitle/video mapping information includes indication information indicating a VOBS 
linked to at least one or more subtitles. Also, information indicating linking relations between 
VOBSs is also included. 

[0063] The subtitle/video mapping information includes at least one among language 
information (lang ="xx") on which language is used to generate the subtitle data, screen display 
title information of a subtitle (caption = "yy"), information on at least one font (font = "zz"), I SRC 
code information of a producer who made subtitles (isrc = "aa"), color table information to be 
used for a background color and foreground color of a subtitle character (palette "pp"), and 
stream number information of a text subtitle (streamed = "id") to distinguish the stream number 
information of the text subtitle from subtitle stream data of a subpicture method in AV stream 
data that is moving picture data. The stream number information of a text subtitle may include, 
for example, a number set to SPRM (2), which is a system register of a reproducing apparatus, 

[0064] FIG. 12 is a diagram showing an example of mapping information shown in FIG. 11 
implemented by using a markup language on a DVD disc. As shown in FIG. 12, video object 1 
in <vmg_vobs> tag is linked to a subtitle data file 1202, file://English_vmgm.text, video object 1 
in <vtms_vobs> tag is linked to a subtitle data file 1204, file://English_vtsm.text, and video object 
1-9 in <vtstt_vobs> tag is linked to a subtitle data file 1206, file://EnglishJt1vob1.text. In 
addition to language information, title information, font information, ISRC information, stream 
IDs, and color table information are included. 

[0065] The example shown in FIG. 12 uses the XML method but other encoding methods 
may also be used for presentation. That is, the data to be used is of primary importance, and 
the method of expressing the data may use a markup language, a binary table, or another 
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method. Data that may be included in subtitle/video mapping information may be arranged as 
follows. 

[0066] FIG. 13 is a diagram showing an example of implementing the mapping information on 
a DVD disc in a form of a table. Referring to FIG. 13, position information 1302,1304, and 1306 
of subtitle data corresponding to a video object is included. In addition to language information, 
title information, font information, ISRC information, color table information, and stream ID 
information may be included. 

[0067] In the DVD, subtitle data to be displayed overlapping a video image is encoded in a 
same manner as 2-dimensional graphic data. That is, in the case of a DVD, a separate 
encoding method to support multiple languages is not employed and data for each subtitle is 
converted into pixel data for a bitmap image graphic, and then the pixel data is processed by 
one encoding method and recorded. Graphic data for a subtitle is referred to as a subpicture. A 
subpicture is formed with a subpicture unit (SPU) corresponding to one sheet of graphic data. 
As output control information to output pixel data, output starting time information, color 
information of pixel data, information on contrast with video data, and information on an output 
time and a finish time are included. 

[0068] FIG. 14 is a diagram explaining the structure of subtitle data shown in FIG. 6 
implemented on a DVD disc. Referring to FIG. 14, the structure of the subtitle data shown in 
FIG. 6 includes header information, reference offset information, time information and/or text 
information. The header information and text information are the same as explained with 
reference to FIG. 6 and a description thereof will not be repeated. 

[0069] However, the reference offset information is reference information to randomly search 
for a subtitle of a desired time at a high speed and reproduce the subtitle, and uses the cell 
identifier (VOBJDN and CELLJDN) of the video object (VOB) described above, which is used 
as a recording unit of a DVD storage medium. Also, as synchronization time information, a 
lapse time from a starting time point of a reference cell is used. In particular, synchronization 
time information may be expressed by using a presentation time stamp (PTS) time based on a 
reference time to reproduce moving pictures. Style information specifying the output style of 
pixel data and control information for controlling output of converted pixel data may also be 
included. The style information and control information will be explained below. 



15 



Docket No.: 0001.1157 
Substitute Specification 

Clean 

[0070] FIGS. 15A and 15B are diagrams showing an example of subtitle data implemented 
on a DVD disc displayed on a screen . 

[0071] In a DVD, pixel data is losslessly coded and information on a position at which a 
subpicture of a subtitle is displayed (SP display area) in an area to display the entire screen, 
and information for controlling an output start time and finish time, an output direction, and line 
spacing are included. In a DVD, subpicture data for subtitle data of a maximum of 32 different 
languages may be multiplexed together with moving picture data and recorded. Each language 
is distinguished by a stream id provided by the MPEG system coding method, and a sub stream 
id defined by the DVD. Accordingly, if a user selects one language, the corresponding 
subpicture unit (SPU) is extracted by taking only subpictures (SP_PCK) having the stream id 
and sub stream id corresponding to the language, and then, by decoding the subpicture unit 
(SPU), subtitle data is extracted and according to output control information, the output is 
controlled. 

[0072] Referring to FIGS. 15A and 15B, a method of displaying the text-based subtitle 
constructed to have compatibility with the subpicture method of a DVD disc is shown. The 
subtitle data described with reference to FIG. 14 is parsed by the subtitle processor 840 of FIG. 
8 and converted into pixel data having an output image as shown in FIGS. 15A and 15B, and 
transferred to the blender 850. Accordingly, the reproducing apparatus shown in FIG. 8 receives 
the pixel data for a bitmap image as the conventional bitmap-based subpicture data, 
synchronizes the pixel data with the video data and outputs an image corresponding to the 
video data and the synchronized pixel data on the screen. As a result, without modifying the 
structure of the conventional decoder 830, a subtitle may be processed to be compatible with 
the conventional subpicture method. 

[0073] The text-based subtitle data described above may comprise the following structure in 
order to be compatible with the subpicture method of a DVD as described above. 

[0074] FIG 16 is a diagram showing an example of subtitle data implemented by a markup 
language on a DVD. Referring to FIG. 16, when text-based subtitle data is randomly searched 
for, in order to search for the data at a high speed the storage medium uses a video object 
number (VOBJDN), a cell number (CELLJDN), and a lapse time from a cell (time) as reference 
offset information (1600). Information indicating next reference offset information (next) is 
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included. Text information including the contents of a subtitle and style information indicating an 
output style are included (1602 and 1606). Control information for outputting pixel data obtained 
by converting a text subtitle after synchronization with video data is also included (1604 and 
1608). 

[0075] In order to generate the pixel data as shown in FIGS. 1 5A and 1 5B based on the 
subtitle data of FIG. 16, style information as the following is included in the text-based subtitle 
data according to the present invention. 

[0076] Style information included in subtitle data may also include information on a width and 
a height of a pixel data area to indicate the size of the pixel data. Also included are information 
on a color of a background screen of pixel data rendering subtitle text, time information on when 
pixel data is to be generated and deleted in the buffer memory of pixel data, information on a 
starting point from which the first line of subtitle text begins, information on line spacing when a 
plurality of lines of subtitle text are output, information indicating whether the output direction of 
text is from left to right or from right to left, or in the vertical or horizontal direction, information on 
the shape, such as size, bold and Italic, of subtitle text, line break information, and character 
code change information. 

[0077] The subtitle processor 830 parses style information in subtitle data having the 
information described above and controls the output style of the subtitle text. In order to control 
so that pixel data converted by applying the style information described above is synchronized 
with video data and output on the screen, the following control information is included. 

[0078] The control information in the subtitle data used to generate the pixel data shown in 
FIGS. 15Aand 15B includes position information ((Xs, Ys), (Xe, Ye)) specifying an area to 
output the pixel data on the screen, information on a starting point in the pixel data area 
corresponding to the first starting point of a text subpicture display area (Text SP Display Area) 
shown in FIG. 15A, and time information on when the bitmap image of pixel data appears and 
disappears on the screen. 

[0079] FIG. 16 illustrates subtitle data information 1600 which, when vobjd is any one of 1 to 
5 and celtjd is 1, a subtitle is output by calculating a synchronization time (sync time) based on 
vob_idn=1-5 and cell_idn=1. That is, by using a lapse time (time) from the video object number 
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(VOBJDN) and cell number (CELLJDN) that are reference offset information, when text-based 
subtitle data is randomly searched for, retrieval may be quickly performed. 

[0080] A <text> tag indicates style information and subtitle text information and a < pixmap> 
tag indicates control information for outputting pixel data. 

[0081] As 'begin' or 'end' information of the <pixmap> tag or the <text> tag, a lapse time from 
a Cell or PTS time information calculated from the standard time clock (STC) described above is 
appropriate. Also, 'start' information of the <text> tag indicates address information of a starting 
point from which text is rendered, and 'start' information of the < pixmap> tag indicates 
information on an address from which the pixel data is to be output. 

[0082] In addition, as an example of control information, synchronization time information for 
outputting subtitle data synchronized with video data may be expressed as a lapse time from a 
reference cell of a video object, and in particular, screen output time (Presentation Time Stamp, 
PTS) information based on the STC used by a reproducing apparatus to reproduce video data 
may be used. 

[0083] The example shown in FIG. 16 uses the XML method. However, the contents of 
subtitle data including style information and control information, reference offset information and 
header information, described above may be expressed by other encoding methods. 

[0084] A case where subtitle data is expressed by a binary table method will now be 
explained. FIG. 17 is a diagram showing an example of subtitle data implemented in a form of a 
binary table on a DVD disc. Referring to FIG. 17, unlike in FIG. 16, an example of subtitle 
implemented in the form of a binary table shape is shown. As shown in FIG. 17, subtitle data 
includes style information 1710, control information 1720, text information 1730, and header 
information indicating starting positions of the style information 1710, the control information 
1720 and the text information 1730. Pixmap #1 (1712) is an example of control information 
1720 and includes information indicating a position of style information (style = T) and a 
position of text information (text_offset = T). Accordingly, corresponding subtitle text 
information may be found and by applying the style information 1710, pixel data may be 
generated and based on the control information 1720, the output of the generated pixel data 
may be controlled. 
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[0085] In particular, by implementing each of the style information 1710 and the control 
information 1720 to have a predetermined size and to be stored sequentially in a predetermined 
area, search time for random search and reproduction is reduced compared to the conventional 
XML structure such as SAMI technology and Real-text technology, such that efficiency of search 
is enhanced. 

[0086] Referring to FIGS. 18 through 24, an example of implementing mapping information 
and subtitle data on a bluray disc will now be explained. 

[0087] In a bluray disc, an AV stream in which moving picture data is recorded is divided into 
clip units and recorded. In addition, according to a presentation method similar to the 
subpicture method of a DVD, subtitle data is converted into pixel data based on a bitmap and 
processed. 

[0088] Implementing the data structure of subtitle data and mapping information shown in 
FIG. 4 on the bluray disc structure will now be explained. 

[0089] Referring to FIG. 18, a storage medium includes a clip that is a recording unit of 
moving picture data of a bluray disc and indication information indicating subtitle data 1802 
linked to the clip. 

[0090] Subtitle/video mapping information includes indication information indicating a clip 
linked to at least one or more subtitles. The subtitle/video mapping information includes 
language information (lang = M xx") on which language is used for subtitle data, screen display 
title information of a subtitle (caption = M yy"), information on at least one or more fonts (font = 
"zz"), ISRC code information of a producer making the subtitle (irsc = "aa"), color table 
information to be used for a background color and a foreground color of a subtitle character 
(palette "pp"), and information on a packet identifier (PID) number of a text subtitle (pid="id") to 
distinguish from subtitle stream data in the subpicture method in AV stream data that is moving 
picture data. The PID number information of a text subtitle can include, for example, a number 
set to PSR (2), which is a Player Status Register. 

[0091] FIG. 19 is a diagram showing an example of mapping information shown in FIG. 18 
implemented by using a markup language on a bluray disc. As shown in FIG. 19, subtitle clip 
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file 0001 in <blu-ray-video> tag is linked to subtitle data file 1902, 'englishj)001.text.' 
Additionally, language information, title information, font information, ISRC information, 
information on the PID number of a subtitle, and color table information are included. 

[0092] The example shown in FIG. 19 uses the XML method but other encoding methods 
may also be used for presentation. That is, the data to be used is of primary importance, and 
the method of expressing the data may use a markup language, or a binary table, or another 
method. 

[0093] Data included in subtitle/video mapping information may be arranged as follows. 
FIG 20 is a diagram showing an example of the mapping information described in a form of a 
table implemented on a bluray disc. Referring to FIG 20, position information of subtitle data 
corresponding to a video clip is included (2002). Additionally, language information, title 
information, font information, ISRC information, color table information, and stream ID 
information may be included. 

[0094] FIG. 21 is a diagram explaining a structure of the subtitle data shown in FIG. 6 
implemented on a bluray disc. As shown in FIG. 21, subtitle data includes header information, 
reference offset information, time information and/or text information. The header information 
and text information are the same as explained above with reference to FIG 6. 

[0095] However, the reference offset information is reference information to randomly search 
for a subtitle of a desired time at a high speed and reproduce the subtitle, and uses reproduction 
time information (presentation time stamp, PTS) of the start position of a clip described above, 
which is used as a recording unit of a bluray disc storage medium. Also, as synchronization 
time information, a relative time to a starting time point of reproduction of a clip is used. In 
particular, synchronization time information may be expressed by using a presentation time 
stamp (PTS) time based on a reference time to reproduce moving pictures. 

[0096] Additionally, style information specifying the output style of pixel data and control 
information for controlling output of converted pixel data may be included. The style information 
and control information are explained below. 
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[0097] FIGS. 22A and 22B are diagrams showing an example of subtitle data implemented 
on a bluray disc displayed on a screen. Referring to FIGS. 22A and 22B, the subtitle data 
described with reference to FIG. 21 is parsed by the subtitle processor 840 shown in FIG. 8 and 
converted into pixel data having an output image as shown in FIGS. 22A and 22B, and 
transferred to the blender 850. Accordingly, the reproducing apparatus receive the pixel data for 
a bitmap image as the conventional bitmap-based presentation data of a bluray disc, 
synchronize the pixel data with video data and output the synchronized pixel data and the video 
data on the screen. As a result, without modifying the structure of the conventional decoder 
830, a subtitle may be processed to be compatible with the conventional presentation method of 
a bluray disc. 

[0098] The text-based subtitle data described above has the following structure in order to be 
compatible with the presentation method of a bluray disc as described above. FIG 23 is a 
diagram showing an example of subtitle data implemented by a markup language on a bluray 
disc. Referring to FIG 23, when the text-based subtitle data is randomly searched for, in order 
to search for the data at a high speed a storage medium uses the reproduction start time point 
(sync-offset value) of a video clip. Also, information indicating next reference offset information 
(next) is included. Also, text information including the contents of a subtitle and style information 
indicating an output style are included (2302 and 2306). Further, control information for 
outputting pixel data obtained by converting a text subtitle after synchronization with video data 
is also included (2304 and 2308). 

[0099] In order to generate the pixel data as shown in FIGS. 22A and 22B based on the 
subtitle data of FIG. 23, style information described with reference to FIG. 16 may be included in 
the text-based subtitle data. The subtitle processor 830 parses the style information in subtitle 
data having the information described above and controls the output style of subtitle text. 

[00100] In order to control synchronization of the pixel data converted by applying the style 
information described above with the video data and to output the synchronized pixel data and 
video data on the screen, the following control information is included. 

[00101] Control information included in the subtitle data may include position information 
(cropping) specifying an area to be actually output on the screen among pixel data, information 
on a starting point (pixmap position) in a pixel data area corresponding to the first starting point 
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of a text subpicture display area (Text SP Display Area) shown in FIG. 22A, and time information 
(composition begin end) on when the bitmap image of pixel data appears and disappears on the 
screen. 

[00102] In FIG. 23, subtitle data information for outputting a subtitle in which a reproduction 
start time of a clip is *0' is shown. That is, by using the reproduction start time information of a 
clip that is reference offset information, when text-based subtitle data is randomly searched for, 
retrieval is quickly performed. 

[00103] Also, <text> tag indicates style information and subtitle text information and 
<composition> tag indicates control information for outputting pixel data. 

[00104] As 'begin' or 'end' information of <composition> tag or <text> tag, a lapse time from a 
reproduction start time of a clip or PTS time information calculated from the standard time clock 
(STC) described above is appropriate. Also, 'start' information of <text> tag indicates address 
information of a starting point from which text is rendered and 'position' information of <pixmap> 
tag indicates information on an address from which pixel data is to be output. 

[00105] In addition, as an example of control information, synchronization time information for 
outputting subtitle data synchronized with video data may be expressed as a relative time based 
on the reproduction start time point of a video clip, and in particular, screen output time 
(Presentation Time Stamp, PTS) information based on a STC used by a reproducing apparatus 
to reproduce video data may be used. 

[00106] The example shown in FIG. 23 uses the XML method. However, the contents of 
subtitle data including style information and control information, reference offset information and 

header information, described above may be expressed by other encoding methods. A case 

i' 

where subtitle data is expressed by a binary table method will now be explained. 

[00107] FIG. 24 is a diagram showing an example of subtitle data implemented in a form of a 
binary table on a bluray disc. 

[00108] Referring to FIG. 24, an example of subtitle data implemented in the form of a binary 
table shape is shown. As shown in FIG. 24, the subtitle data includes style information 2410, 
control information 2420, and text information 2430, and includes header information indicating 
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starting positions of these information items. The detailed structure is explained above with 
reference to FIG. 17. 

[00109] In particular, by implementing each of the style information and control information to 
have a predetermined size and to be stored sequentially in a predetermined area, search time 
for random search and reproduction is reduced compared to the conventional XML structure 
such as SAMI technology and Real-text technology, such that efficiency of search is enhanced. 

[00110] The present invention can also be embodied as computer readable codes on a 
computer readable recording medium. The computer readable recording medium is any data 
storage device that can store data which can be thereafter read by a computer system. 
Examples of the computer readable recording medium include read-only memory (ROM), 
random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, optical data storage 
devices, and carrier waves (such as data transmission through the Internet). The computer 
readable recording medium can also be distributed over network coupled computer systems so 
that the computer readable code is stored and executed in a distributed fashion. 

[00111] As described above, by using the mapping information and the subtitle data file 
structure according to the embodiments of the present invention, subtitle information compatible 
with a bitmap image method such as the subpicture method of a DVD and the presentation 
method of a bluray disc, and based on text corresponding to moving picture data having a 
plurality of playback routes can be provided. 

[00112] By including reference offset information in a subtitle data file, when random search or 
reproduction is performed the time for searching for a subtitle may be reduced such that search 
efficiency is enhanced. 

[001 1 3] Although a few embodiments of the present invention have been shown and 
described, it would be appreciated by those skilled in the art that changes may be made in this 
embodiment without departing from the principles and spirit of the invention, the scope of which 
is defined in the claims and their equivalents. 
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